Parametric array modulation and processing method

ABSTRACT

A system using the nonlinearity of a propagation medium to demodulate an ultrasonic wave having audio modulated onto the ultrasonic frequency, audio signal processing circuitry providing a delayed audio signal and an envelope signal which is a function of peaks of the audio signal over a predetermined interval. The delayed audio signal and the envelope signals are combined with the delay in the audio allowing the envelope signal to accurately be aligned with the audio signal in the combination.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application incorporates by reference prior commonly ownedU.S. Applications 60/185,235 filed Feb. 28, 2000; 60/406,230 filed Aug.26, 2002; and Ser. No. 09/300,033.

FIELD AND BACKGROUND OF THE INVENTION

The parametric array uses the nonlinear response of a transmissionmedium such as air or water to convert or demodulate ultrasonicfrequency waves into audio frequency waves of any audio frequencysignals modulated onto the ultrasonic waves. This phenomenon is usefulto direct a beam of ultra sound having audio modulated thereon to aspecified region where it is demodulated in the medium and can be heard.

In a typical case of a parametric array in air, the audible signal willtypically be from voice, music or other normal audio frequency source.Prior to or subsequent to the conversion or modulation into theultrasound frequency ranges, some form of signal processing is typicallyundertaken. This is undertaken to compensate for the non-flat frequencyresponse of typical ultrasound transducers, the transducer nonlinearity,environmental conditions of temperature and humidity and the position ofthe listening recipient among other effects that prevent a faithfulreproduction of the original sound to the listener. The response of airto ultrasound is also nonlinear and may need compensation prior to theactual ultrasound emission.

The form of modulation typically employed provides a signal envelope onan ultrasonic carrier. The carrier and envelope have different responsecharacteristics to the type of signal processing and the non-flat andnonlinear characteristics of the environment and devices used in thistype of ultrasonic sound beaming from source to listener and a resultinglack of alignment.

SUMMARY OF THE INVENTION

The present invention uses an envelope summed with the audio signal andan envelope detector to supply an adjusting offset to the source audiosignal, such that the envelope of the audio signal, when added to theaudio signal, is entirely positive (or entirely negative). When this isthe case, a nonlinear preprocessing method can be applied (such astaking the square root, or other nonlinear function) accurately. Inaddition, residual sound generated by the envelope signal should beinaudible in the resulting demodulated beam.

A preferred way to accomplish this would be to “look ahead” at the audiosignal, to see what the (peak) values will be some time in the future,and begin adjusting the envelope signal well in advance of a change.Because a processing system cannot actually look into the future theinvention either estimates the signal based on past knowledge or allowsome small delay between the input and output signals. Using an audiosignal delay, it is possible to anticipate the audio signal, and tochange the envelope signal accordingly so that the sum conforms andremains positive. The result is an envelope signal which faithfullyfollows the peak levels of the audio signal, but changes only gradually(ensuring only very low-frequency residue).

DESCRIPTION OF THE DRAWING

This and other features of the invention are more fully described in thedetailed description below in conjunction with the Drawing of which:

FIGS. 1A-1D are waveform diagrams on the summed audio and envelopeuseful in understanding the invention;

FIG. 2 is a block diagram illustrating the practice of the invention;

FIG. 3 is a block diagram illustrating signal processing providingcompensation for characteristics of using ultrasound audio projectionaccording to the invention;

FIGS. 4A-4D illustrate signal processing associated with FIG. 3circuitry; and

FIGS. 5A-5C illustrate further signal processing associated with FIG. 3circuitry.

DETAILED DESCRIPTION

The generation of ultrasound from audible sound will in general requiresome step of modulation, which is simply a re-scaling of frequency. Wecan write this modulation step as:p(t)=M(t)sin ωtWhere M(t) can be termed the modulation envelope, and ω is the carrierfrequency. Basic parametric array theory predicts that, upondemodulation, the resulting audible sound q(t) is approximatelyproportional to the second derivative of the square of the modulationenvelope:q(t)∝d²/dt²′(M²(t))This function is an approximation, but serves well to illustrate theideas contained herein.

We can define an arbitrary preprocessing function P{z}, which acceptssome signal (primarily) in the audio range as input, and outputs aprocessed signal suitable for modulation oat the output. Thus, wegenerally have M(t)=P{z}, where z is some function of g(t). Note thatthe algorithm P{x} may also accept inputs such as environmentalcondition, listener position, desired sound quality, etc.

Early parametric arrays used simple AM modulation to generate theaudible signal, using M(t)=(1+mg(t)), or P{z}=z, where g(t) is theaudible signal to be reproduced (assumed to be normalized to unitymagnitude), and m is the modulation depth, usually taken as one orslightly less than one. The resulting sound generated is:q(t)∝d ² /dt ²(M ²(t))=2md ² /dt ²(g(t))+m ² d/dt ²(g ²(t))

We can see that the resulting sound contains the desired linear termg(t) (we will now omit the second derivative, it being a simpleequalization step and simple to compensate by integrating g(t) twice).We also have a nonlinear term mg²(t), which corresponds to distortion.If we require m to be small, the distortion will be reduced, but thecorresponding output signal will also be reduced by the same factor,which is undesirable.

An improved method is to use P{z}=z½, where z=1+mg(t), leading toM(t)=(1+mg(t))½. Upon demodulation, the resulting audible signal isproportional to mg(t). While this is the result we seek, there are twodrawbacks to this method. First, taking the square-root operation of asignal results in the generation of a substantial set of harmonics,which increase the required bandwidth of the ultrasonic transmissionsystem. If the bandwidth of the transmission system is insufficient toreproduce the entire ultrasonic signal, distortion will result. This wasinvestigated theoretically in [1] and experimentally in [2].

The preprocessing function P{z}=z½ is a reasonably effective method ofpreprocessing the audible signal at low modulation frequencies and lowultrasonic amplitude. However, to improve performance, P{z} should bealtered to more accurately model the true nonlinear modulation function.The particular algorithm is described elsewhere (see referencedprovisional), but we can generalize the function with a nonlinearpiecewise polynomial function, and perhaps a linear filter.

A major shortcoming of having the argument z=1+mg(t) is that when noaudible sound is intended to be reproduced (g(t)=0), the modulationfunction M(t) is unity (M(t)=1). This means that the system is stilloutputting high levels of ultrasound, which is not being used to createaudio, as the output signals is still p(t)=sin ωt.

To alleviate this latter shortcoming, it has been proposed to use amodulation envelope which contains an envelope follower. This iscommonly implemented as a rectifier and low-pass filter. The detectedenvelope of the audio input signal is intended to be a faithful followerof the amplitude of the input signal, although with some time delay.Adding this envelope to the audio signal can provide a suitable offsetwhich keeps the signal positive, allowing an accurate preprocessingoperation:z(t)=e(t)+g(t)M(t)=P{z}=(e(t)+g(t))^(1/2)q(t)∝d²/dt²(e(t)+g(t))q(t)∝d²/dt²(M²(t))

The audible signal g(t) (with second derivative omitted) is reproducedas before, and there is a residual term consisting of the secondderivative of the audio envelope e(t). As long as this frequency is low(recall that it is the result of a low-pass filter), it should notreproduce substantial distortion components. In general, we wish thefrequency of the envelope e(t) to be lower than about 100 Hz.

FIG. 1A illustrates the problem of this type of envelope detector inthat the envelope curve 12 in following the audio signal at a cut off inthe vicinity of 100 Hz isn't a good peak follower and is at times belowthe audio signal.

The main goal of the envelope detector is to supply an appropriateoffset to the incoming audio signal, such that the envelope, when addedto the audio signal, is entirely positive. When this is the case, anonlinear preprocessing method P{z} can be applied (such as taking thesquare root, or other nonlinear function) accurately. In addition,residual sound generated by the envelope signal e(t) should be inaudiblein the resulting demodulated beam. The slow changing function e(t)cannot accurately keep up with a generally fast-changing dynamic audiosignal g(t).

An elegant solution according to the invention is to “look ahead” at theaudio signal, to see what the (peak) values will be some time in thefuture, and begin adjusting the envelope signal well in advance of achange. Because our processing system must be causal (we cannot actuallylook into the future), we may either guess at the signal based on pastknowledge, or, even easier, allow some small delay between the input andoutput signals. If the audio signal is delayed, there is opportunity toanticipate the signal, and begin to change the envelope signalaccordingly. The result is an envelope signal which faithfully followsthe peak levels of the audio signal, but changes only gradually(insuring only low-frequency residue).

A block diagram showing this method employed in discrete time (as in aDSP) is shown in FIG. 2. As the processing my be digital or analog,hardware of software based, it is to be under stood that the circuitdescription applies as hardware modules of processing steps in thefollowing description.

The input (audio) signal x[n], 16, which may have been processed in aprocessor 18 (i.e. equalized, filtered to remove all low-frequencycontent, etc.) is split onto two paths 20 and 22. The signal on path 20is first rectified in a rectifier module or step (assuming DSP) 24, orother envelope detection, to determine its magnitude. The peak value ofthis rectified signal is tacked in a module or step 26 over the previousM samples, represented by p[n]. This peak signal p[n] is then low passfiltered in a filter 28, generally a very low frequency, and the resultis the envelope e[n]. The raw signal, x[n] on path 22 is delayed indelay module or step 30 by N samples or a predetermined interval, whicheffectively corresponds to the settling time (or group delay) of the lowpass filter 28, plus any other delay present in the signal path. Thisensures that the envelope e[n] is properly aligned to the audio signalx[n]. Finally, the envelope signal and audio signal are summed in summeras shown. The result is an accurately offset signal, which is alwayspositive, and is suitable for nonlinear preprocessing (such assquare-rooting) through P{z} and modulation.

As a variation, since we are primarily concerned with keeping x[n]positive, we need only concern ourselves with tracking the negativepeaks of x[n]. Thus the absolute value or rectifier function could bereplaced with an inverter (−x), and the peak detector, rather thanlocating the maximum of M previous samples of x[n], would locate andtrack the minimum (maximally negative) samples of x[n].

FIG. 1B and FIG. 1C which is a magnification of a portion 34 in FIG. 1Bshows the effective of this processing where envelope and signalfunctions are brought into alignment. FIG. 1D shows the ratio of thesignal level to the noise level for such an approach as a function offrequency and illustrates acceptable levels of S/N values.

FIG. 3 illustrates a complete system from audio input to sound waveoutput and applies generally to either hardware or software realization.The x(n)+e(n) output of the FIG. 2 processing is optionally applied toan upsampling and low pass filtering module or step 40 which improvesthe available bandwidth for use in a subsequent preprocessing module orstep 42 characterized by the function P(z). The preprocessing function,more fully described below, is generally nonlinear and functions atleast in part to compensate for the nonlinearities in the ultrasoundgeneration and demodulation functions. The sampling rate of function 40is preferably sufficiently high so that the harmonics inherent in thenon linear processing do not alias and create unintended distortion.

The preprocessed signal may subsequently be upsampled and low passfiltered in module or step 44 after which it is modulated in modulatoror step 46 onto an ultrasonic frequency carrier from carrier generatoror step 48.

The modulated signal is then optionally post processed in a module orstep 50 which may include equalization to compensate for frequencydependent variations in the transfer functions of subsequent ultrasonicamplifiers 52 and transducers 54 or nonlinear processing to compensatefor non linear transfer functions in these same elements 52 and 54.Other processing may be added here to accommodate environmental aircharacteristics or phased array phasing. Alternatively the modulatedand/or post processed signal could be converted into pulse widthmodulated waveform or the like for driving amplifiers 52 where they areswitching amplifiers.

The preprocessing module or step 42 in the simplest form consistent withproviding a low distortion audio demodulated signal has a square rootfunction. Because the nonlinear nature of the preprocessing generatesharmonics and because the subsequent amplification and transductionfunctions have limited bandwidth, other approaches such s a polynomialexpansion of the type P(z)=a₀+a₁z+a₂z² . . . , the a's being functionsof the particular environmental and processing system characteristics.More sophisticated processing could be a polynomial with coefficientsthat are non zero for specific value ranges or other forms of series.

The processing of a monotone, shown in FIG. 4A, at the point ofpreprocessing to give E(t) is shown in FIG. 4B. This createssingularities near or at the zero crossing which contributes to a largebandwidth due to the high values of the derivatives at the abruptreversal. One approach to curing this problem is to modulate E(t) with abipolar squarewave, F(t), as shown in FIG. 4C which produces the lowbandwidth signal, E(t)F(t) of FIG. 4D.

This approach can be modified for use with real signals of unpredictablefrequency content by reversing the bipolar signal F(t) of based on anestimating function in the preprocessor 42 causing the polarity reversalwhen one of the following criteria are met:

i. E(t) proximity to zero;

ii. Magnitude of derivatives of E(t) are high (either first or higherorder derivatives);

iii. E′(t) zero crossing from negative to positive (i.e. E′ is zerowhile E″ is positive);

iv. A short-time power spectrum analysis is made and the result shows ahigh bandwidth.

FIG. 5A shows the operation when the signal envelope, E(t), nears zeroshowing how E(t)F(t) functions to provide a smoothing function. In thecase where the polarity reversal occurs when the envelope signal E(t) isnear but not at the zero crossing as shown in FIG. 5B, a discontinuitywith consequent perturbances can occur. To rectify this situation,preprocessor 42 can add in a spline segment as shown in FIGS. 5B and 5Cto produce the smooth transition from E(t) to E(t)F(t).

The features of the can be realized in alternative, equivalent ways. Forexample, either changing the carrier level directly or the offset of theaudio signal are mathematically substantially equivalent and thusfunctionally equivalent.

1. A system using the nonlinearity of a propagation medium to demodulateultrasonic waves having an audio signal modulated onto the ultrasonicfrequency, comprising: audio signal processing circuitry including:delay means for the audio signal providing a delayed audio signal;envelope generator means providing an envelope signal which isresponsive to negative peaks of the audio signal over a predeterminedinterval; and combiner means for the delayed audio signal and theenvelope signal, the resulting combined signal being useful inprocessing for modulation of said ultrasonic frequency; andpremodulation processing means for processing the combined signalincluding the delayed audio signal and the envelope signal, therebyallowing the propagation medium demodulation to provide a demodulatedacoustic signal which is a substantially accurate representation of theaudio signal.
 2. The system of claim 1 wherein at least one of saiddelay means and said envelope generator means comprise analog circuitry.3. The system of claim 1 wherein at least one of said delay means andsaid envelope generator means comprise digital circuitry.
 4. The systemof claim 3: wherein both said delay means and said envelope generatormeans comprise digital circuitry; wherein means are provided forproviding digital sampling of said audio signal, thereby providing adigitized audio signal; wherein said delay means delays N samples ofsaid digitized audio signal; and wherein said envelope generator meansexamines M prior samples of said digitized audio signal.
 5. The systemof claim 4 wherein N and M are set at values to align the digitizedaudio signal to corresponding times in the envelope signal.
 6. Thesystem of claim 1 further including a low pass filter for the envelopesignal and having a settling time or group delay where a delay intervalcorresponds to a settling time or group delay of the low pass filter. 7.The system of claim 1 wherein said premodulation processing meansgenerates an approximate square root function on the combined signal. 8.The system of claim 1 wherein said premodulation processing meansprocesses said combined signal by a polynomial expansion of apredetermined number of terms.
 9. The system of claim 1 wherein saidpremodulation processing means processes said combined signal by use ofa precalculated lookup table.
 10. The system of claim 1 wherein saidpremodulation processing means includes upsampling and low pass filtermeans to provide an enhanced bandwidth prior to premodulationprocessing.
 11. The system of claim 1 further including up sampling andlow pass filter means prior to any modulation.
 12. The system of claim 1wherein, in response to the negative peaks of the audio signal, saidpremodulation processing means provides for dynamic polarity reversal ofthe combined, processed signal prior to modulation at one or morespecified times within a predetermined interval, thereby reducingbandwidth of the modulated ultrasonic frequency.
 13. The system of claim12 wherein said one or more specified times corresponds to one or moreof criteria that the unmodulated, processed signal as applied to thepremodulation processing means is: close to a zero value; has arelatively high slope; has a short-time power spectrum estimate thatindicates a wide bandwidth; and has a slope that is near a zero valuewhile a rate of change of the slope is positive.
 14. The system of claim1 further including means for ultrasonically modulating the combinedsignal.
 15. The system of claim 14 further including means forprojecting ultrasonic sound wave representations of the modulatedcombined signal.
 16. The system of claim 15 wherein said projectingmeans includes amplifier means and transducer means.
 17. The system ofclaim 16 further including means for providing an offset bias in themodulated signal.
 18. The system of claim 17 wherein said offset biasmaintains the modulated signal in a predetermined polarity.
 19. In asystem using the nonlinearity of a propagation medium to demodulateultrasonic waves having an audio signal modulated onto the ultrasonicfrequency, audio signal processing circuitry comprising: envelopegenerator means for tracking negative peaks of the audio signal over apredetermined interval and inverting the negative peaks, therebyproviding an envelope signal which is an approximate function of theinverted negative peaks of the audio signal over said predeterminedinterval, said approximate function having misalignment of the envelopesignal and the audio signal; and means for converting the audio signaland the envelope signal into an ultrasonic signal characterized by acarrier signal and reduced misalignment.
 20. The audio signal processingcircuitry of claim 19 wherein said converting means includes means fordelaying the audio signal.
 21. The audio signal processing circuitry ofclaim 19 wherein said converting means includes means for adjusting thelevel of said carrier signal to reduce said misalignment.
 22. The audiosignal processing circuitry of claim 19 wherein said means forconverting includes means for providing for polarity reversal of theunmodulated, combined, processed signal at one or more specified timeswithin a predetermined interval in response to negative peaks of theaudio signal, thereby reducing frequency bandwidth of the modulatedultrasonic signal.
 23. In a system using the nonlinearity of apropagation medium to demodulate ultrasonic waves having an originalaudio signal modulated onto the ultrasonic frequency, an audio signalprocessing method comprising the steps of: delaying the original audiosignal to provide a delayed audio signal; generating an envelope signalwhich is responsive to negative peaks of the audio signal over apredetermined interval; combining the delayed audio signal and theenvelope signal to produce a combined signal useful in processing formodulation of said ultrasonic frequency; and in a processing step,processing the combined signal including the delayed audio signal andthe envelope signal, thereby allowing the propagation mediumdemodulation to provide a demodulated acoustic signal which is asubstantially accurate representation of the original audio signal. 24.The method of claim 23 wherein said processing step includes the step ofadjusting the level of a carrier signal to increase tolerance formisalignment of the envelope signal and the audio signal.
 25. The methodof claim 23 wherein said processing step includes the step of providingfor polarity reversal of the combined, processed signal prior tomodulation at one or more specified times within a predeterminedinterval in response to the negative peaks of the audio signal, therebyreducing bandwidth of the modulated ultrasonic frequency.